1. Field of the Invention
This invention relates to the field of digital signal processing and, more specifically, to digital processing of image signals obtained from a multi-color image sensor.
2. Description Relative to the Prior Art
There are practical limitations on the sophistication of signal processing in an analog video system, which in turn limits the image quality such a system is capable of producing. Digital signal processing offers a technique for more sophisticated processing, thus offering the promise of higher image quality. In designing a fully digital camera system, the fundamental structure of the system is ordinarily based on transposing the elements of a conventional analog video camera system from the analog to the digital domain. This, for example, is seen in the digital system proposed by Nikoh et al ("The Full Digital Video Camera System and Simulation of Its Essential Parameters," by H. Nikoh and T. Kuwajima, Digest of Technical Papers, International Conference on Consumer Electronics, June 1988, pp 4-5). Most processing is done digitally, including pixel error correction, color separation, white balance, gamma correction, color difference signal generation, aperture compensation and noise reduction. A similarly comprehensive digital architecture is seen in EPO patent application No. 289,944 (published Nov. 9, 1988), wherein an electronic digital still camera performs color separation, white balance adjustment and gamma correction prior to storing the processed signals in a digital memory. Such direct transpositions from analog to digital ordinarily involve a quantization space in which the quantization increments are linearly related to the sensor signal level.
There are occasions, however, when it may be advantageous to do the digital signal processing in other than linear space. For instance, in Ser. No. 07/290,729, filed Dec. 27, 1988 in the name of Easterly et al and entitled "Optical Image to Video Transfer System Having Enhanced Resolution And Contrast For Dark Areas of the Image, (now U.S. Pat. No. 4,912,558) "which is assigned to the assignee of the present application, the analog pixel values produced by a single chip, multi-color CCD image sensor are digitized and converted into a log exposure space, that is, digital code words are generated that have a logarithmic relationship to the input signal amplitudes. The interpolation of all colors is then carried out in log space. Certain operations, however, such as black level correction, must be completed in a space linearly related to the sensor signal level. This is the case because misadjustment of the black level can manifest itself as a color shift if the transfer functions of the sensor and the linear-to-log conversion are mismatched. Consequently, in Ser. No. 07/290,729, the black level of the analog sensor signal is corrected before it is digitized.
The arrangement of the processing architecture is complicated by the fact that certain operations, like multiplication, are inconvenient to implement in a linear digital process. Color signal interpolation requires multiplications or divisions, but these operations are usually by known coefficients that can be implemented, or at least closely approximated, by hard-wired shifts and adds. Other operations, like white balance, involve scene-dependent multiplications that cannot be readily hard-wired into a linear digital circuit. These operations are more readily accomplished in log space as additions.
It would appear desirable to partition the digital process according to the space domain most suitable for the individual operations. This alone, however, does not insure a flexible and streamlined digital architecture since the priority of operations may involve numerous transformations between spaces. For example, while interpolation (itself best done in linear space) follows linear black level correction, the separated chrominance colors should be white balanced (in log space, because of multiplications) before the chroma is interpolated. Moreover, certain operations, like gamma correction, edge enhancement, and color correction might better be done in either space depending on the nature of the correction. What is more, such operations are ordinarily limited to certain applications-such as a particular form of output-and thus impact upon flexibility of the digital process.